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SOFTWARE AND METHODS FOR AUTOMATED PALLET REPAIR 



Field of the Invention 

5 The present invention relates generally to the repair of wooden pallets, and specifically to 

an automated process for scanning pallets and identifying individual elements of the pallet 
for removal, replacement, or repair. It is also applicable to pallets constructed from other 
materials such as plastic, metal or composites. 

1 0 Background of the Invention 

Commercial movement of materials typically use a wooden pallet on which the material is 
placed or secured. This pallet is typically constructed with a flat upper deck consisting of 
planks or boards of timber nailed, screwed, or glued to parallel beams known as bearers or 

15 stringers. Bottom boards are similarly attached to the bearers. The framework allows the 

insertion of the "forks" of a forklift or other machine to raise and move the pallet and its 
load of materials. There are several pallet designs in use and are distinguished generally 
by place of manufacture and use. For example, pallets made and used in Australia, New 
Zealand, the United States, Canada, and Europe are all of different designs. In some 

20 designs, blocks are used with or in place of bearers to separate the top and bottom boards. 

During normal use, the pallets may be dropped, overloaded, crushed or otherwise 
damaged. Damaged pallets are often returned to the pallet provider or pooler for 
inspection, repair or replacement. 

25 

The inspection and decision process is currently done by skilled human inspectors or by 
automated means that implement the specific criteria, and decide to repair or discard a 
damaged pallet. Using human inspectors is desirable because they can inspect and 
immediately repair each pallet at a single station. This can be done by presenting each 
30 damaged pallet in turn, for example, on a conveyor, such that the inspector can see pallet 

damage and repair it if necessary. It is undesirable because the inspection and repair 
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decision is not uniform as each inspector will naturally implement repair based on his or 
her judgement. It is also undesirable from a safety point of view as accidents will happen 
in such an environment. 

5 Current automated systems use stereoscopic pairs of cameras to collect pallet geometry and 

topography information, then use computer programmes to make a repair / discard decision 
by uniformly implementing the specific pallet criteria. This is done by first detennming 
the pallet's design, e.g. Australian or European, and comparing the geometry and 
topography of the individual pallet against criteria for the pallet's design. Current systems, 

10 however , decide only to repair or discard each inspected pallet, that is, each pallet either 

does or does not pass the inspection criteria. If the pallet passes, it is placed back in 
service. If it does not pass, it is sent for repair. The repair process, however, is similar to 
the manual inspection process above. The pallets needing repair are sent, for example, by 
conveyor, past one or more human repair stations where the repairer inspects the pallet, 

15 determines what needs repair and then repairs it This has the same disadvantages in 

unifoimity and safety of the human inspection process. It is faster, however, since the 
human inspectors are seeing only those pallets determined to need repair. 
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Objects and Summary of the Invention 

It is an object to provide an automated process which not only inspects pallets and 
determines when to repair or discard, but also identifies the pallet element or elements that 
need to be removed, replaced, or repaired, and passes this additional information to a 
human or automated repairer. 

It is therefore an object of the present invention to automatically inspect pallets, identify 
the pallet design, compare that geometry and topography of the pallet against pre-defined 
quality criteria, determine when to repair or discard, identify specifically the element or 
elements needing repair and the exact nature of the repair needed. 
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It is another object of the present design to provide the repair data to an automated repair 
process, thus eliminating both human inspectors and human repairers. This makes the 
process of pallet repair faster, safer, and more uniform than current methods. 

5 In summary of one embodiment, without being considered a limiting example, a pallet to 

be inspected is moved relative to an inspection head. This may be on a conveyor or using a 
robotic manipulator or other device. Alternatively the pallet may be in a stationary 
location and the inspection head may move across it In preferred embodiments, the 
sensing head consists of a set of at least one laser and a camera, with the camera recording 

10 the projected laser profile across a width of a pallet Extra cameras can be used to help scan 

wider areas but stereoscopic camera pairs are not needed. Resulting information from the 
sensing head is collected and processed to represent the geometry and topography of the 
pallet A two dimensional representation of the pallet is analysed so that the individual 
elements are identified and located by coordinates. The pallet design is determined by the 

15 number, size, and location of the elements. The elements are analysed against specific 

criteria for the pallet's determined design. This includes criteria for the element alone 
(size, location, integrity, damage, missing or raised nails, etc.), inter-elemental criteria 
(spacing, overlap, etc.), and pallet design criteria (missing or superfluous elements, etc.). 
If the pallet is determined to have not passed the criteria, a recipe or Ust of specific repairs 

20 is generated. This list includes which element is to be repaired and the nature of the repair 

(remove, replace, reattach, repair, etc.). The data comprising the list of repairs to be made 
accompanies the pallet to the repair station. In preferred embodiments, the repair station is 
an automated repairer, for example, a robot arm using a nail gun, band or other saw, prying 
levers, etc., to implement the exact repairs determined necessary. After repair, the pallet is 

25 returned to service. 

In preferred embodiments the present invention provides coordinate outputs sufficient to 
automate, through robotic arm for example, the component repairs. 
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Brief Description of the Drawings 

Figure 1 is a schematic top plan view diagram of a pallet; 
Figure 2 is a schematic top plan view diagram of an inspection station; 
5 Figure 3 illustrates a schematic side elevation of a pallet; and 

Figure 4 illustrates a logic flow chart of a pallet repair process. 

Detailed Description 

10 Figure 1 illustrates one design of a pallet 1 00. The pallet consists of top boards 102, 

labelled TB 0 through 7, and corresponding bottom boards labelled BBO through BB4. 
The top boards are supported by three horizontal bearers or stringers 104, 106, and 108, 
labelled B0, Bl and B2. Other designs will have different numbers of boards, different 
sized boards, and different spacing between boards, and may have different numbers and 

1 5 styles of bearers. Blocks and connector boards may be substituted for bearers in some 

designs. In the illustrated design, top board 0 and top board 7 are wider than the other six 
boards. For the purpose of this invention the pallet can be considered to be laid out in an 
orthogonal x y z configuration where the x axis runs horizontally (with reference to 
Figure 1) across the bottom of the pallet parallel to beam or bearer 108. The y axis runs 

20 vertically (with reference to Figure 1) along the left edge of top board 1. The z axis is 

orthogonal to both x and y. 

There are four processes which make up the preferred method of automated pallet 
inspection and repair. Each will be discussed below. The first step is data capture. This 

25 means capturing all of the data about the physical or structural make up of a pallet which 

is required to make the determinations about the nature, extent and order of the repair 
process. The next step is the analysis of the captured data. The analysis simplifies the 
data and relates the data to known facts so that a repair process, using certain fixed 
processes can be specified. Next is a process of generating a recipe or fist of process 

30 based on what repair processes are available and what repairs are needed based on the 

data and analysis of it After recipe generation is the last major step, that of the 
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implementation of the recipe using automated equipment In preferred embodiments, an 
industrial robot reads a step specified by the recipe and implements that according to a 
flexible programme before implementing the next step of the recipe. 

5 The present invention consists of (for example) two computer systems and a mechanical 

means for moving pallets. These systems may reside on one or more physical computer 
processors or hardware, and may be distributed or collected. The first computer system is 
called the capture system, which collects information about the geometry and topography 
of the pallet. The second computer system is called the analysis system, which analyses 

10 the geometry and topography of the pallet and determines the pallet design and then using 

the pallet design specifications analyses the pallet and decides whether it is to be repaired 
or returned to service. It is not required that the tow computer systems be separate or 
distinct The mechanical means moving the pallet may be a conveyor belt or a robotic 
arm or other system for transporting Ihe pallet through the inspection head. 

15 

Figure 2 illustrates one implementation 200 of the mechanical means for moving pallets. 
This comprises a chain conveyor supporting a pallet 204 and moving it in the direction of 
the arrow, left to right The pallet 204 moves under twin lasers 206 and 208. These 
lasers munnnate and sensors capture the entire width of the pallet 204. In some 

20 embodiments there is an overlap in Ihe laser beams in the middle as shown by 2 1 0. As 

the pallet 204 moves under the lasers 206 and 208, information about the geometry and 
topography of a pallet is captured by cameras and sent to the capture computer system 
212. Such laser and sensor systems are well-known, as are the methods for using such 
lasers to collect this information. The system may be replicated to collect data on the 

25 other faces of the pallet simultaneously or asynchronously from the top deck. It will be 

obvious that this may also be achieved using a different number of laser and camera 
combinations than the twin set up described above. 

The capture computer system 212 performs the following steps. First it collects the 
30 profile information of the pallet that'is, it collects the geometry and topography 

information from the lasers and sensors. The sensors return a stream of three- 
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dimensional coordinates. The cameras/sensors are synchronised so that overlapping 
points illuminated by multiple lasers give the same coordinate values when viewed from 
each sensor or camera. The scans from the two lasers are then combined to give a set of 
5 coordinates for the entire width of the pallet. This process is repeated for each profile 

scanned as the pallet moves relative to the lasers and cameras 

When analysing the top deck, the scanned data is filtered to give only the top surface 
geometry and topography. This is accomplished by discarding any points which have a 
10 z-coordinate which is below a given threshold or filter line. This removes from analysis 

any point corresponding to a bearer or the bottom of the pallet or the transporting 
conveyor or robotic manipulator, for example. It will be obvious that the same process 
could be applied when analysing any specific face or deck of the pallet 

15 In some embodiments, individual planes are established for each bearer. The planes can 

be combined, averaged or used as separate data references. 

The laser scans are timed according to the speed of the pallet transport mechanism so that 
scans occur at regular distances along the length of the pallet in the direction of 
20 movement. Typically, this is set to scan at 1mm linear distance, though it could be at any 

chosen resolution distance. 

Next, the comer points of the pallet are found using a 45 degree filter. This locates the 
four points at the extremities of the pallet. That is, it finds the point of minimum x 
25 minimum y, minimum x maximum y, maximum x minimum y, and maximum x 

maximum y. These four points determine the comers of the pallet. These are typically 
referred to as PPO, PPl, PP2, and PP3 respectively, where PPO and PP2 lie on the x-axis, 
and PPO and PPl lie on the y-axis. PPO and PP3 are diagonally, as are PPl and PP2. 

30 Next, it finds the offsets between the image origin and pallet origin to give the x and y 

offset distances of the pallet. That is it calculates the size of the pallet by subtracting 
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combinations of PPO, PP1, PP2, and PP3. The data is also normalised by relocating the 
coordinates so that PPO lies at the origin of the pallet coordinate system, and PP1 and PP2 
lie on the x- and y-axis respectively. A second set of coordinates, based around the image 
datum, is used when calculating automatic repair parameters. 



To convert from the three dimensional topographical information to a two dimensional 
geometric representation, first the locations and heights of the bearers (labelled BO, Bl, 
B2) are found in the image by inspecting the profiles most likely to represent bearer 
10 locations. 

When the bearer heights and locations have been determined, a series of filter planes is 
drawn offset from the bearers (shown as element 302 in Figure 3). Each point in the 
three dimensional representation is then checked against the corresponding filter plane. 

15 Points above the filter plane are identified as belonging to boards, points below belonging 

to bearers or other structural elements. The board points are then further filtered and 
assembled into arrays of points on board edges belonging together. This can be done 
using any standard edge finding technique applied to the set of points above the filter line 
and an edge chain following algorithm. These edge arrays represent boards or parts of 

20 boards and are used in later analysis. If a two dimensional (geometry only) scanning and 

inspection head is used, electronic means (eg sensor range restrictions) are used to filter 
the boards from other data, with the same array identification and assembly process 
taking place. The arrays do not contain height related data, only 2D geometrical position 
of points above the filter line. 

25 



The analysis computer system 214 performs the steps shown in Table 1 . 
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TABLE 1 



Step 


Description ■ — — 


1 


The data stream from the inspection head is captured to construct a pallet model in 
computer memory. This process uses the known resolution o the inspection neaa ana me 
known pallet velocity and distance from the sensing head to construct a three dimensional 
topographic model and subsequently a two dimensional geometric model of the pallet bemg 
analysed. Pallet corner points are calculated from topographical data and stored. 


2 


Geometric model is broken down to give arrays of points representing the edges of each 
board (and edges of partial boards) 


3 


Board arrays are checked for completeness (i.e. is the edge a closed loop?) and consistency 
(i.e. edges do not cross) Joined boards are split by applying a virtual edge along the most 
probable line of intersection between the boards. Where there are multiple arrays along a 
single line drawn parallel to the Y axis and these arrays are all less than a full board length 
in the Y direction and not overlapping or intersecting, they are classed as a single (broken) 
board. Board arrays are then sorted by their minimum X value, and their comer points are 
identified and stored. 


4 


By comparing the number, type and location of identified boards and the separation 
distance between pallet comer points with the range of possible known pallet geometries, a 
pallet type can be assigned to the pallet undergoing analysis, deterrnined by the closest 
match against the database of specifications. This matching process can be accelerated if 
the system need only expect a single pallet style. A pallet that can not be matched to a 
specific pallet style is marked as undefined and further analysis on it is halted. 


5 


Board types can be assigned (eg Intermediate, Lead, etc) to each board based on its location 
relative to pallet corner points and its approximate width (from board comer points). 
Boards within a specified region of the pallet comer points are presumed to be lead boards. 


6 


Pallet quality criteria are loaded from the database into the analysis system for the particular 
pallet type determined above. Each board array can be checked against the appropriate 
board criteria for that board type. Board checks may include board width, notches or 
missing material, jagged edges, excessive crookedness, or any other criteria. Any board 
array that fails these tests is marked as aboard to be removed. Topographic data for the 
region corresponding to the board is also checked for board thickness, end spUtting, cracks, 

_ . , .« « jj MAM MA«ni faahiiw \irifV\ -fa 1 1 1 iff* c n(Tain hp.inf* recorded for removal, 
holes and other three dimensional teatures, .wim iauures agoui u6u, b x ^ 


7 


Examine the board arrays against other quality criteria to determine if other non-removal 
board repairs are necessary. An example would be the position of the lead boards relative to 
the pallet comer points, with any lead board that is too far from the comer points marked to 
be adjusted, unless it has already been marked for removal. A hierarchy of board repair 
decisions is imposed with the board removal the highest precedence, board realignment 
next, and any other operation lowest. 


8 


Gaps are checked against gap criteria, such as gap width. Gaps that are larger than a board 
width are checked to see if a board will allowably fit into the gap with appropnate resulting 
gaps on either side (Fig 4). If a board will fit, a phantom array is constructed to represent , , 
the missine board, and it is marked for a board placement operation. 


9 


All results are stored in the database. At this point, all arrays are marked as eitner valid 
boards or as boards to be removed, adjusted or operated on. 
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10 


When this system is implemented in an automated pallet repair situation, further 
calculations and data manipulation are required. These calculations are specific to each 
machine in the repair cell, and might include the location (X, Y&Z position and angles) 

j 4~ j^ioAY-f o Vkonriooxi/ V\1qHp» ititn a narrieiilar onn to nerform the removal operation 

for a board that has been marked for removal. Where this blade does not fit into the gap, 
the board is marked for removal with a different device. 


11 


A recipe is generated for the repair cell, with a list of operations Gobs) required to be 
performed to repair the pallet, and the associated data for each of these jobs. These are 
sorted to expedite the repair cycle time 


12 


In an inspection system designed for sorting or for quality control purposes only, steps 10 
and 1 1 are removed, and replaced by further topographic analysis of protruding nails and 
other features. 



Fig. 4 illustrates the logic flow for step 8 in Table 1, the process for examining the gaps 
400. The software defines the storage arrays to hold gap values to use for each of the 

5 profiles 402. Each gap is initialised to zero. Starting with the right hand edge of the left 

most board, the gap values are calculated for each board as shown in step 404. In step 
406 the gap values for each board have been stored, so the average gap can be calculated, 
m step 408 the average gap is compared against the criteria for gap based on the pallet 
design. If the gap is larger than the design criteria, the gap is marked as a bad gap, shown 

10 in step 418. In step 420, the bad gap is examined to see whether it is large enough to fit a . 

new board. If it is large enough to fit in a board, step 424 calculates how many boards 
will fit into the gap. That number of boards is then indicated for the repair orders. The 
software then moves to step 422 to examine the next gap. However, if the decision made 
at step 420 is that the gap is not wide enough to fit in the new board and it still exceeds 

15 the criteria for the maximum gap, then step 428 is performed. Step 428 determines which 

boards must be removed and replaced to fix the gap. 

In step 430 a check is made to see if one of the bounding boards is crooked. If the 
bounding boards are crooked, the offending board is indicated for removal and 
20 replacement or repositioned, and the resulting gap is re-evaluated 426. If none of the 

bounding boards are crooked 430, then a check is made to see if one of the boards is 
missing any wood (or other material for non-timber pallets) 432. If one of boards is 
missing material, 432, an order is indicated to remove the board and re-evaluate the 
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resulting gap in step 426. In step 432, if no boards are missing any material, then step 
434 is performed. A check of the neighbouring gaps is made. If one of the neighbouring 
gaps is smaller than the other then an order is indicated to remove the board and re- 
evaluate the resulting gap step 426. If in step 434 the gaps are equal size then step 436 is 
performed, that is, the pallet is marked for manual inspection, or alternatively a decision 
can be made to arbitrarily remove one of the boards. 

With reference to step 408, the average gap is compared against the design criteria, and if 
the average gap is acceptable, a test is made to determine if there is a notch in the board. 
Such a notch would give a false indication of a bad gap. The notch test is shown in steps 
410 through 416. At 410, the gap values across the notch length are added and the 
average calculated. In step 416, the calculated average is compared to the design criteria. 
If the average is greater than the design criteria and the gap is too big processing 
continues to step 418 described above. If at step 416, the average gap passes the test 
against the design criteria that a check is made in step 414 to determine if there are more 
gap values to check. If there are more to check, the step 412 is performed to add the next 
value and then subtract the first value and recalculate the average. Processing then 
continues and step 416. If at step 414 there are no more gap values to check, then it has 
been determined that all gaps values are acceptable and processing continues at step 422 
to move to the next gap. If there are no more gaps to test, then processing ends at step 
438. 

Returning now to step 426, a repair order indicates the removal of a board or the 
reposition it. Processing then continues at 404 to recalculate the average gap. 

In this way, the pallet is examined board by board and repair orders are stored for later 
use or the pallet is returned to service. If the pallet needs repair, specific instructions are 
determined for removing, replacing, repositioning boards, or to add one or more boards, 
or to remove a protruding nail. 
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This provides a technical advantage over current automated pallet inspection and/or 
repair systems, which only determine a pass-fail decision for the pallets suitability, and 
no specific repair instructions are generated. In addition, the technique of the present 
invention is sufficient to automate the repair process by connecting the output of the 
5 process to an automated repair station. Such a station could comprise a robot arm which 

grasps the pallet to be repaired, then, using a band saw and nail gun and other devices, 
removes and replaces specified boards. The instructions to the robot arm would be to, for 
example, "remove the board located between 22.5 cm and 40 cm from the leading edge, 
then nail a new board at 22.0 cm from the leading edge." 

10 

This same logic may be appUed to the inspection of and repositioning of or replacement 
of the bearers, or alternatively to the lower deck of the pallet. 



15 Controlling the robot in an automated repair cell based on the information generated by 

the pallet analysis system described above requires specific robot, PLC and computer 
system linkages or interfaces and software; 

Traditional robot control design is relatively simple, based around the premise that the 
20 robot performs a repeatable job (or series of repeatable jobs) that can be predefined at the 

time the system is designed. Further, traditional robot systems require human 
intervention when there is a problem or a robot crash. To use a robot for pallet repair, it 
must dynamically change its program for each pallet to be repaired, based on the 
particular operations required and the location of the boards or gaps to which these 
25 operations must be applied. Further, it must automatically recover from problems and 

minor crashes, and flag major crashes to an operator. 

To achieve this, the software system for cell control is broken into three components - 
these are the robot controller, the programmable logic controller (PLC) and the repair 
30 recipe generation sub-system described in Steps 1 0 and 1 1 of Table 1 . The recipe 

generation sub-system loads the necessary robot operations and associated position data 
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into the PLC. This can be done in a single batch for a whole pallet, or sequentially as 
required. In preferred embodiments the data is sent as a single batch. Data is checked for 
consistency and completeness, and any missing data is flagged to the recipe generation 
system. 

The robot controller contains a master job and a series of sub-jobs that the master job can 
call when required. The master job communicates with the PLC. The PLC tells the 
master job on the robot controller which sub-job to call, and sends it the data that this 
sub-job will need to run, for example the location and angle of a particular board to be 
operated on. This data is confirmed by the master job to the PLC, with the master job 
waiting for a handshake from the PLC before continuing. On receipt of the handshake, 
the sub-job is called. The first task in each sub-job is to check the robot's current 
location. The position and angle of the pallet gripper must be within a certain envelope 
(which may be a sphere or a cylinder around a predefined point or line known as the sub- 
job home position) for the operation to continue. If the robot is within the allowable 
envelope, the sub-job continues with the operation. At each step of the operation, a 
handshake is exchanged with the PLC. This handshake allows the PLC to monitor the 
point in the sub-job that the robot is up to, which is required for any automatic recovery 
operations. When the sub-job is finished (or if an error is encountered) control of the 
robot is passed back to the master job, which communicates with the PLC to get the next 
sub-job, and the process continues until the recipe is complete. 

Each device in the repair cell is numbered, starting with device 02 and working up to 
device 99 (this system could be extended to use any number of digits depending on the 
number of devices in the cell, however two digits are used in preferred embodiments). 
Robot sub-jobs that occur at each of these devices are named after the device where they 
occur, eg a two digit code. The sub-jobs that control travel between machines are named 
by combining the names of the two devices between which the robot must move (eg the 
sub-job for moving from machine 12 to machine 34 would be 1234, whereas to move 
from 34 to 12 would be 3412). These numbers are generated by the repair recipe sub- 
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system and passed as part of the recipe. The PLC then passes these names in turn to the 
robot master job, which then calls the matching sub-job. 

Some numbers are reserved for emergency or other recoveries, namely devices 00 and 
01. This allows recovery sub-jobs to be defined for each device as XX00 and XX01 
where XX represents the device name. Recovery jobs generally reverse out through the 
steps previously performed at that machine, back to the robot home position for that 
device (eg if sub-job 03 runs into trouble, the recovery job would be 0300 or 0301, 
depending on the alarm generated). Depending on the alarm condition, the current recipe 
step is tried again, or the alternatively the recipe is changed dynamically to overcome the 
problem. For example, if the removal of a board to be removed with device 03 fails, and 
device 04 is also designed to remove boards, extra steps can be added to the recipe 
dynamically to take the pallet to device 04 and perform the removal operation. 
Depending on the particular recipe, this action may take place immediately, or after the 
completion of any other pending operations at device 03. 

The system relies on the PLC being in ultimate control at all times, and for handshakes 
between the robot and the PLC between any operations, no matter how small. For safety 
purposes, all robot operations must check their start position and confirm this with the 
PLC. 

An alternative embodiment of the recipe system would be in a repair cell where the pallet 
is held in a single location and the inspection head and repair devices are brought to it 
The pallet analysis would proceed as per the earlier description, as would the recipe 
generation. In this style of embodiment, rather than the PLC instructing the robot to take 
the pallet to a particular repair device, the PLC instructs the robot (or other manipulator) 
to bring the repair device to the pallet The transfer of position data is identical to the 
above description, as are the handshaking and robot location checking procedures. 
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